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ABSTRACT 

A fundamental fact of the analysis of variance 
statistical procedure is that if the cftibus P tes^ of an effect is 
significant^ then there exists at least on€ contrast of ttat effect 
that will be significantly different fici leio according to the 
S-iethoi of Scheffe. The caveat to this rule is that the significant 
contest (s) lay not be of any interest to or inter pretable by the 
inv^^^iqator. Frustration does arise when the investigatcr detenines 
the oinibus F test to b^ significant and th^n fails tc find any 
Meaningful group conparisons to be sigrificant. The purpose of this 
paper is to discuss the coefficients of a contrast which will yield 
the largest test statistic according to the S-pethod. Thet according 
to the fundanertal theorem, this contrast will be significantly 
different from zero and, additionally, should offer the irvestigatcr 
some assistance in developing a parsiicricus interpretation of the 
data. The S-method is a post-hoc method of comparing experimental 
groups and is often referred to as a data -snooping or sifting devicel 
The technigues discussed here are also of the £9ta-snoopir g species 
and may ba employed to generate new hypotheses cr to amecd the 
existing theory of a practical problem. (Author/CTH) 
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The Coefficie nts of a Maximum. Contrast as Interpretable Statistics 
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Introduction ; ^ ^ 

/■ [ ^ ' 

A fundamental fact of the analysis of variance statistical proce^re 

^ ■* ■ 

is that if the omnibus F test of an ^ffect 16 significant, then there exists 
at lea^t one contrast of that effect that wilA be significantly different 
from^^ero according to . the ^-method of Scheffej (^19^3, p. 70).* ,^e caveat \, 
to this rule is that the significJint contrast (s) may not.be of any interest 

to or interpretable by tUe investigator. Frustration does arise when the 

' ' >■ ^' . ^ ■ « 

investigator determines the omnibus F test to be significant and th*n fails 

W 

to find any meaningful group comparisons (usually pairwtse comparisons) 
to be significant. The purpose of this paper is to -discuss the coefficients 
of a contrast ^^^^ will yield the l^irgest test statistic according to the 
S-method. Th%rl- according to the fundamental, theorem, this contrast will be 
significantly different from zero and, addttlonally , should offer the. . 
investigator someir assistance in developing a parsii^nious Interpretation of 

.the data. , . * * y 

V 

It is well known that the S-m^Bod is a post-hoc method of comparing 



experiemtal groups and is ^f ten referred to as a'' data-snooping or \sifting 
^ device (Keppel, 1973, p. 93). -, The techniques to be discussed h^ire are k^so 

of data-snooping species and may be employed to generate new hypotheses 

or to amend the existing theory of a practical problem. 



*paper presented at th^/Annual Meeting of the Am^ican Educational Research 
Association, March, 1978. ^ 

er|c / ^ 



* 



Method 

To ease the discussion, let us consider a one-way analysis of variance 

situation with a treatment groups and _s observat^ns in each group. Let 

A and T be . the mean observation in group i and the grand mean, respectively. 

A contrast among the population means u , . . . , y is a litiear 9ombination 

1 * ' 'a 

a , ^ a < 

i); - L c w where E c - 0 and at least two of the c 's ar^ non-zero. 
i-1 ^ ^ i-1 ^ ' 



The unbiased estimate of is ij; - I c A . 

i-1 ^ 



The statistic SS(A comp) - s 



a 

-ft cj, defined by Keppel (1973, p*98), 
i-1 



can be thought as the sum of squares due ta the comparison of an effect A 
defined by a contrast. SS(A comp) has a chi-square distribution with one 
degree of freedom and is independent of the statistic, mean square within 
groups, MS(W) . 

Since ''ss (A comp) is bounded frbm above by the sum of squares betweeq 

groups, SS(A), (Keppel, 1973, p. 109), we will define a maximum contrast, 

i , to be a contrast for which SS(A comp) - SS(A). The existence of ' 
max ^ Bifx 

is discussed by Scheffd (1959, p. 71) and a set of coefficients of ^^^^ 

given by Winer (1971, p. 176). * A set of coefficients sligl>tly different from 
Winer's, will now be introduced. 

One set of coefficients, {c^*}, of a maximum contrast is {j/s - T) / yJSS(A) } 

It is not necessary to show the<^niJ^trhematical derivation of these^ coefficients, 
for it-is not a difficult task to verify that this set satisfy the requirements 
of being a contrast, as defined, and of maximizing the sum' of squares due to 
the contrast. As ^poiat of interest, one derivation of the coefficients 
wuld bcr'^siiijfiar'^ that^f Scheff^ (1969, p.ll8)^j^re the notion of 
Lagrange multipliers ie^itillsed. 
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The major difference bet)^eert Winer 'ft aet and the above set of 

cdefficients allows for a further interpretation of the data. For the 

set of coefficients presented in this pap^r, the square of a coefficient, 
2 

c^, is the proportion of the between groups' variation attributable to group ^ 

This follows from tiie equality c^ - (\ ~ T)^ / E (A - T)^, i.e. is 
' ) i-1 ^ ^ 

a 

the ratio of a part to the whole. Note for the set, {c }, 2: c^ • i. 

1 i-1 ^ 

Example • ^ 

In a study reported by Derevensky (1976), sixty kindergarten children 
were asked to perform a memory task oi placing colored sticks in a specific 
order under six different instructional conditions. The subjects were 
instructed to either watch and/or listen to an experimenter manipulate the 
sticks and then replicate the manipulation starting with Vn identical or a 
different spatial arrangement of the sticks. The meah f or each of the six 
cqnditions are given in Table 1. It is assumed that the design was balanced, 



i.e. s - 10. 



Table 1 

MEANS FOR EACH CONDITION AND THE COEFFICIENTS OF THE MAXIMUM CONffl^ST 

Condition-Spatial Arrangement X c 

-^-^ , i i 



1. Visual-only-identical j' 2.10 -.3706 . IsVa 

2. Verbal-only-identical / 2.90 .2224 .0495 

3. Visual-verbal-identical V 3.00 .2965 .0879 

4. Visual-only-different 1.60 -.7412 .5495 

5. Verbal-only-d^fferent 3.00 .2-965 |o879 

6. Visual-verbal-different f 3.00 .2965 .0879 ^ 
TOTAL 2.60 



A one-way analysis of variance of the data revealed a significant 
instructional condition effect. However, simple pairwise comparisons of 
the groups were all judged to be s^tatistically non-significant at the .05 level 
using the ^-method. 



Table 2 





ANALYSIS 


OF VARIANCE 








Source 


df 


SS 


MS 


F 




Conditions 

Error 

TOTAL 


5 
54 
59 


18.20 
52.20 
70.40 


3.64 
.97 


3.77 




0.05 


The coefficients of 


4> and 
max 


their squares 


are reported 


in Table 


1. 



An observation of these coefficients suggest that performance under the 
visual only instructional conditions (conditions 1 and 4) are inferior to 
the other instructional conditions. The same interpretation would be made oT 
the basis of a subjective inspection of the maans (Table 1)., However, 
interpretations based on the coefficients of th^ maximum contrast allow for 
a natural progression to post-hoc data analyti(V* procedures . For example, it 
would seem natural to test the post-hoc hypothesis, 

.5(uj^ + u^) - .25(u2 + M3 + U5 + u^) - 0, whiclK would be considered as 
contrasting or comparing the visual only confiitjions with the other four 
instructional conditions, and this hypotheslVioaid be rejected at Ve .05. 
level o^ significance. One might also want to injtyrpret the significantly 
different from zero (p<.05) contrast, which approximates tjie maximum contrast,' 
(.3u^ + .7u^) - .25(u2 + U3 + U5 + u^). This contrast ^ug^ests that the 
visual only conditions produce a different efSect than^'the other fouW 
instructional conditions and, furthermore, the spafial arranfement does 
differentially effect the .perfoniance of' the subjects within the visual only ^ 



condition. 

ons under tne Instructional eoryjition 

2 



The observations under the instructional eoryjition ntober 4 is accounting 
for nearly 552 (c^ - .5495) of the between groi^p var^itipn as defined by SS(A). 
So that if the observations under group .4 w6reTemoved from the analysis. 



the between groups sum of squares would be more thai) halved and the omnibus ' 
F test may no longer be significant at the desire^level^Of significance. 



Discussion ^ 



One advantage of post-hoc comparisons is th^t TthV^xpe^menter may 
uncover an interesting, but \0planned, comparidpn wftlch cduld be incorporated 
into the existing theoretical framework ^f^he research area and used to 
generate further hypotheses, thi^ respect , it is obligatory that the 



experimenter analyze the maxlMwr.contrast , New research, then can be 

deslg^d test hypotheses sugge|i:ed byN^he maximum contrast and existing 

theory can be modified to reflect \an observed ^'tate of nature. 

When the experiment has une^j^a^ jtample ilNes, then the defining 

formula for the sum of sqiS^res between groups will differ depending on the 

metTiod of Analysis (Keppf 1,^^1973, pp, 346-47), If one is using the unweighted 

means analysis, then .the^toef f icients of i^*"" "^vou^ be {/s(A - T)/7 SS(A) } 
^ 'J- max i 

where s is the harmonic mean of the sample sizes of the groups and 

T - lAja. 

A guneralizatiwi of this discussion of the maximum contrast can be 
made to factorial desig^|^. The reader is referred to Chapters 11 and 12 of 
Keppel (1973) for a treatment of multiple comparisons in a factorial experi- 
ment with two factors. It is natural to ask ±f ^ maximum contrast exists 
that could be used to est;|^te the degree to which an experimental cell 
contributes to the irtteraction of two or Wore factors, TJie answer is 
affiVmative and that contrast will be discussed at a later date. 

Finally, a comment about/generalizability of the method is in order. 
The coefficients of a maximum contrast are computed from the observations 



made on a sample. At be8t;, these coefficients should be considered the same 
as regression coefflfcienta with respect to their ability to estimate population 
characteristics. To* avoid the errors of over interpretating the results of. 
an experiment, it is suggested that new data be collected to validate, 
using a-priori comparison procedures, the interpretations based on a max;Lmum 
contrast. . 

REFERENCES 

DereVensky, J», Developmental -changes in the effect of verval, nonverbal, and 

spatial-positional cues for memory. ^ The Journal of Experimental Ed ucation 
1976, 45^, 52-60., ^ ^ 

Keppel, G>,r Design and Analysis: A Researchers Handbook , Englewood Cliffs: 
Prentice Hall, Inc., 1973. 

Scheff^, H., The Analysis of Variance , New York: John Wiley & Sons, Inc., 1959 

Winer, B.J., Statistical Principles in Experimental Design , New York: 
McGraw-Hill, 1971. 



8 



